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Abstract. In everyday scenes, the illuminant can vary spatially in chromaticity and luminance, 
and change over time (e.g. sunset). Such variation generates dramatic image effects too 
complex for any contemporary machine vision system to overcome, yet human observers 
are remarkably successful at inferring object properties separately from lighting, an ability 
linked with estimation and tracking of light field parameters. Which information does the 
visual system use to infer light field dynamics? Here, we specifically ask whether color 
contributes to inferred light source motion. Observers viewed 3D surfaces illuminated by an 
out-of-view moving collimated source (sun) and a diffuse source (sky). In half of the trials, 
the two sources differed in chromaticity, thereby providing more information about motion 
direction. Observers discriminated light motion direction above chance, and only the least 
sensitive observer benefited slightly from the added color information, suggesting that color 
plays only a very minor role for inferring light field dynamics. 
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The human visual system is remarkably successful at disentangling lighting from object properties in 
the ambiguous light signal entering our eyes. Several studies show that observers estimate both surface 
properties and light field parameters from static views of 3D scenes (see Maloney, Gerhard, Boyaci, & 
Doerschner, 2011, for a recent review). We have shown that observers also estimate dynamic light field 
properties in 3D scenes, an ability linked with improved sensitivity to surface reflectance (Gerhard & 
Maloney, 2010a ). Analyses showed that perceiving light field dynamics depends heavily on the 3D 
surface geometry of the scene's objects, much more so than on the raw luminance patterns in the im- 
ages (Gerhard & Maloney, 2010b ). However, previous dynamic light stimuli were achromatic. Here, 
we ask whether color improves sensitivity. 

Following previous studies, our subjects viewed rendered 3D scenes stereoscopically. These stim- 
uli simulated a sun setting over a hilly landscape ( Figure la ) viewed from above ( Figure lb ). Mo- 
nocularly, peaks and valleys cannot be disambiguated in such scenes so lighting direction is inferred 
correctly only up to a 180° flip, and disparity must be used to avoid the concave/convex ambiguity. 
Trials were 1,500 ms: the initial 750 ms were static so observers could acquaint themselves with the 
geometry and initial lighting, in the final 750 ms, the out-of-view collimated light source, the "sun," 
rotated 10° overhead in one of the four directions ( Figure lc ). The task was 4AFC to indicate the sun 
motion direction. Feedback was never given, and there was no training. 

The sun's initial position above the scene was randomized on every trial both in azimuth (angle 
in the image plane) and in elevation, held between 90° (perpendicular to the ground plane) and 70° 
(slightly off-perpendicular). Random initial positions forced observers to track light motion because 
position in any one frame was uninformative for the task. 

An additional fixed light source also illuminated the scene: a homogeneous "sky." The sky was al- 
ways neutral in hue and a quarter intense as the sun, which had intensity 100 cd/m 2 . The lights' intensi- 
ties were fixed, but we varied their chromaticity coordinates. In half of the trials, both had chromaticity 
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Figure 1. Hilly landscapes and lighting, (a) An example landscape seen from a view never used experimentally 
but shown for illustrative purposes, (b) Example stereo pairs for crossed and uncrossed fusion. Subjects viewed 
the scenes stereoscopically from directly overhead as shown. Monocular views do not disambiguate hills from 
peaks; only stereo disparity resolves the concave/convex ambiguity and therefore the direction to the light, (c) 
The sun rotated 10° over the landscape in one of the four directions in the image plane. The task was 4AFC to 
indicate motion direction, (d) In half of the trials, the sun had chromaticity coordinates equal to one of the eight 
hues equally spaced on a circle in CIE La*b* space centered on D65. The hue with * has CIE x, y coordinates 
(0.375, 0.425). In the other half of the trials, the sun was D65. The sky was always D65. (e) A hilly landscape 
illuminated by a yellowish sun and neutral sky. 

coordinates corresponding to D65, the standard neutral daylight hue (achromatic condition). In the 
other half, the sun had one of eight hues (chromatic condition) shown in Figure 1(d) . Figure 1(e) shows 
a chromatic scene. In total, 1,216 trials were pre-rendered: 608 achromatic and 608 chromatic (76 per 
hue). Blocks alternated by condition, with first block type counterbalanced over subjects. Hue trial 
types were intermixed in chromatic blocks. 

A new random Gaussian landscape was generated for every trial. We ensured that shading was 
the only cue to light direction by assuming Lambertian reflectance and disallowing combinations of 
sun trajectories and scene geometry causing cast shadows since they are an unambiguous cue to light 
direction even monocularly. We rendered 1 6 frames for each trial, in which the sun rotated in incre- 
ments of 0.625°. The experiment was written using the Psychtoolbox (Brainard, 1997 ; Pelli, 1997 ) and 
run on a stereoscope of two linearized monitors. All details of the equipment, scene generation, and 
stereo rendering process are identical to those described in Gerhard and Maloney ( 2010b ). 

Eight subjects participated. All had normal or corrected-to-normal acuity, passed the Ishihara 
color plate test, and passed a stereo test emphasizing concavity/convexity judgments from Gerhard 
and Maloney ( 2010a ). All gave informed consent before participation and were remunerated $8/hour. 
Experiments were approved by the New York University Institutional Review Board and were in ac- 
cordance with the Helsinki Declaration. 

Results are shown in Figure 2 . Observers performed above chance overall, achieving between 
31% and 77% correct after a correction for guessing; however, color information did not significantly 
improve sensitivity to light motion direction ( Figure 2a ) except for the least sensitive observer who 
improved by 7%. Observers were equally sensitive to all motion directions ( Figure 2b ) and for all hues 
( Figure 2c ). 

Added color information does not appear to play a large role in sensitivity to light field dynam- 
ics. Previous analyses emphasizing the importance of scene geometry provide much more explana- 
tory power for this kind of light field inference in hilly scenes illuminated by a moving sun and sky 
(Gerhard & Maloney, 2010b ). 
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(a) Average motion discrimination (corrected for guessing) 
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Figure 2. Results, (a) Motion discrimination performance with 95% confidence intervals for each subject 
(corrected for guessing, chance = 0). Only the least sensitive observer improved significantly (by 7%) with added 
color, (b) Sensitivity did not depend on motion direction. We calculated d' for each motion direction separately by 
randomly splitting each subject's data into four sets of 304 trials. Average d' values ±1 SEM are plotted by motion 
direction (Up, Down, .Right, Left), (c) Sensitivity did not depend on hue. Average percent correct (corrected for 
guessing, chance = 0) is plotted with colored 95% confidence intervals at corresponding locations in CIE La*b* 
space. 



Acknowledgments. We thank Ed Vul for his help in designing and implementing the stimulus generation 
procedure. This work was supported by NIH/NEI EY08266. 



References 

Brainard, D. H. (1997). The Psychophysics Toolbox. Spatial Vision, 10, 433^36. ^ 

Gerhard, H. E., & Maloney, L. T. (2010a). Detection of light transformations and concomitant changes in 
surface albedo. Journal of Vision, 10(9): 1, 1-14. doi: 10. 1 167/10.9. 1 M 

Gerhard, H. E., & Maloney, L. T. (2010b). Estimating changes in lighting direction in binocularly viewed three- 
dimensional scenes. Journal of Vision, 10(9): 14, 1-22. doi: 10. 1 37 l/journal.pone.0054549 ^ 

Maloney, L. T., Gerhard, H. E., Boyaci, B., & Doerschner, K. (2011). Surface color perception and light field 
estimation in 3D scenes. In L. R. Harris, & M. R. M. Jenkin (Eds.), Vision in 3D environments (pp. 
280-307). Cambridge, UK: Cambridge University Press. ^ 

Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. 
Spatial Vision, 10, 437-442. doi: 1 0. 1 1 63/1 56856897X00366 < 



Holly E. Gerhard holds a PhD from the Cognition and Perception program of the NYU Psychology 
Department and is currently a post-doctoral research scientist at the Computational Vision and Neuro- 
science Lab in Tuebingen, Germany. 



Prof. Laurence T. Maloney holds a PhD in Psychology and an MS in Statistics from Stanford University. 
He is currently a core faculty member of the NYU Psychology Department and the NYU Center for 
Neural Science. 



Copyright 201 3 H E Gerhard, L T Maloney 

Published under a Creative Commons Licence I fe) rmratm l Jfc^ a Pion publication 



